Cognitive Capacity and Choice under Uncertainty: Human Experiments of Two-armed Bandit Problems

The two-armed bandit problem, or more generally, the multi-armed bandit problem, has been identified as the underlying problem of many practical circumstances which involves making a series of choices among uncertain alternatives. Problems like job searching, customer switching, and even the adoption of fundamental or technical trading strategies of traders in financial markets can be formulated as the variations of the multi-armed bandit problem. As a canonical representation of the learning problem under uncertainty, the multi-armed bandit problem involves a tradeoff between exploration of gaining additional information of the uncertain alternatives and exploitation of the alternative which has the seemingly higher expected payoff. In light of its significance, a lot of work has been done to provide the optimal or near optimal solutions, such as Jones (1975), Wahrenberger, Antle, & Klimko (1977), and Gittins (1979). However, not much has been done to investigate how human actually solve bandit problems. Early human experiments conducted by Goodnow and Robillard in 1951–1953 reveal a great deal of interesting properties of human behavior in two-armed bandit problems. For instance, in their results, the costs of the exploration behavior may influence how people choose the arm with higher winning probability, and people exhibit inconsistent sensitivity to differences in probabilities of the two arms. 4 Goodnow and Robillard’s experiments in fact raised more questions than they had answered. For example, what strategy do people adopt when facing a two-armed bandit problem? Do they apply Bayesian rules or simply follow certain heuristics? What factors may play roles when people are deciding what kind of strategies to use? 4 Goodnow and Robillard’s experiments are reported in Bush & Mosteller (1955).